Statistical methods for spoken dialogue management

Statistical methods for spoken dialogue management Blaise Thomson Spoken dialogue systems provide a mechanism for interacting with computers that is both natural and effective for human use. This thesis describes a practical framework for building these systems based on the Partially Observable Markov Decision Process (POMDP). The underlying belief state is represented by a dynamic Bayesian Network and the policy is parameterized using a set of action-dependent basis functions. Tractable real-time Bayesian belief updating is made possible using a novel form of Loopy Belief Propagation with various other optimisations and policy optimisation is performed using an episodic Natural Actor Critic algorithm. Parameters for the system’s user model may be learned from an unannotated corpus of dialogues using the Expectation Propagation algorithm with tied Dirichlet distributed parameters. Details of these algorithms are provided along with evaluations of their accuracy and efficiency. The proposed POMDP-based architecture was tested using both simulations and a user trial. Both indicated that the incorporation of Bayesian belief updating significantly increases robustness to noise compared to traditional dialogue state estimation approaches. Furthermore, policy learning worked effectively and the learned policy outperformed all others on simulations. The learned policy was also competitive in user trials, although its optimality was less conclusive. Parameter learning was tested by evaluating the resulting user models’ rescoring ability and by simulating dialogue performance. A range of metrics were shown to improve when using Expectation Propagation to learn a set of parameters and then re-score semantic hypotheses, as shown on both a simulated and a human-generated corpus. Success rates of a system where the parameters were trained from a corpus of dialogues were also shown to outperform the use of handcrafted parameters. Overall, the proposed framework was shown to be a feasible and effective approach to building real-world POMDP-based dialogue systems.